Information processing device, communication system, and information processing method

ABSTRACT

There is provided an information processing device including an operation acceptance unit configured to accept a user operation related to an image based on image data received from another information processing device using short-range wireless communication when the image is displayed on a display unit, and a controller configured to perform control in a manner that notification information for issuing a notification that the user operation is accepted is output when the user operation is accepted, and control data corresponding to the user operation is transmitted to the other information processing device.

CROSS REFERENCE TO RELATED APPLICATIONS

The present disclosure is a continuation of U.S. patent application Ser.No. 15/209,459 filed in USPTO on Jul. 13, 2016 which is a continuationof U.S. patent application Ser. No. 14/072,347 filed on Nov. 5, 2013,which claims priority benefit of priority from Japanese PatentApplication No. JP 2012-247981 filed in the Japan Patent Office on Nov.12, 2012. The above-referenced applications is hereby incorporatedherein by reference in its entirety.

BACKGROUND

The present technology relates to an information processing device, andmore particularly to an information processing device, communicationsystem, and information processing method in which various types ofinformation are exchanged using wireless communication.

In related art, wireless communication techniques in which various typesof data are exchanged using wireless communication have been known. Forexample, there has been proposed an information exchange device in whichvarious types of data are exchanged between two wireless communicationdevices via short-range wireless communication (for example, refer toJapanese Unexamined Patent Application Publication No. 2008-278388).

SUMMARY

According to the above-described technique in related art, various typesof data can be exchanged between two wireless communication devices viashort-range wireless communication without using a wire line. However,when various types of data are exchanged between two wirelesscommunication devices in this way, there may be a delay due toshort-range wireless communication.

In this context, as an example, in a reception-side informationprocessing device that displays an image based on image data transmittedfrom a transmission-side information processing device on a displayunit, it is assumed that the user performs an operation input forperforming an operation related to the image. In this case, it isassumed that a time until an image based on image data reflecting theoperation input is displayed on a display unit is delayed by the amountof delay. In this way, as the time until an image reflecting theoperation input is displayed becomes long, it is difficult for the userto recognize quickly whether the operation input is properly accepted,and thus an erroneous operation such as performing again an operationinput is more likely to occur.

An embodiment of the present technology has been made in view of such asituation. It is desirable for the user to easily recognize anacceptance condition of a user operation.

According to a first embodiment of the present technology, there isprovided an information processing device including an operationacceptance unit configured to accept a user operation related to animage based on image data received from another information processingdevice using short-range wireless communication when the image isdisplayed on a display unit, and a controller configured to performcontrol in a manner that notification information for issuing anotification that the user operation is accepted is output when the useroperation is accepted, and control data corresponding to the useroperation is transmitted to the other information processing device. Asa result, in a case where an image based on image data received fromanother information processing device using short-range wirelesscommunication is displayed on a display unit, when a user operationrelated to the image is accepted, notification information for issuing anotification that the user operation is accepted can be output, andcontrol data corresponding to the user operation can be transmitted tothe other information processing device.

Further, according to the embodiment, the controller may cause thenotification information to be output by superimposing an additionalimage indicating that the user operation is accepted on the imagedisplayed on the display unit. As a result, the additional imageindicating the acceptance of the user operation can be displayed bysuperimposing it on the image displayed on the display unit.

Further, according to the embodiment, the operation acceptance unit andthe display unit may include a touch panel. The user operation may be acontact operation or a proximity operation to the touch panel. Thecontroller may cause the additional image to be displayed by using aposition in the touch panel at which the contact operation or theproximity operation is accepted as a reference when the contactoperation or the proximity operation to the touch panel is accepted. Asa result, the additional image can be displayed by using the position inthe touch panel at which the contact operation or the proximityoperation is accepted as a reference when the contact operation or theproximity operation to the touch panel is accepted.

Further, according to the embodiment, the controller may cause a rippleimage to be displayed as the additional image when a pressing operationof pressing an operation image on the touch panel is accepted, theripple image being centered on a position in the touch panel at whichthe pressing operation is accepted. The controller may cause an imageindicating a trajectory of a tracing operation in the touch panel to bedisplayed as the additional image when the tracing operation of tracingan operation image on the touch panel is accepted. As a result, when thepressing operation of pressing an operation image on the touch panel isaccepted, it is possible to display the ripple image which is centeredon the position in the touch panel at which the pressing operation isaccepted. Moreover, when the tracing operation of tracing an operationimage on the touch panel is accepted, it is possible to display an imageindicating a trajectory of the tracing operation in the touch panel.

Further, according to the embodiment, the controller may cause theadditional image to be erased at a predetermined timing. As a result,the additional image can be erased at the predetermined timing.

Further, according to the embodiment, the controller may determine thepredetermined timing based on a delay time occurred in the short-rangewireless communication with the other information processing device. Asa result, the predetermined timing can be determined based on the delaytime occurred in the short-range wireless communication with the otherinformation processing device.

Further, according to the embodiment, the controller may regard a timingof receiving data indicating that the control data is accepted from theother information processing device as the predetermined timing. As aresult, the timing of receiving data indicating that the control data isaccepted from the other information processing device can be regarded asthe predetermined timing.

Further, according to the embodiment, the controller may regard a timingof receiving image data corresponding to image data identificationinformation from the other information processing device or a timing ofdisplaying an image based on the image data as the predetermined timing,the image data identification information being included in dataindicating that the other information processing device accepts thecontrol data. As a result, the timing of receiving image datacorresponding to image data identification information included in dataindicating that the control data is accepted from the other informationprocessing device or the timing of displaying an image based on theimage data can be regarded as the predetermined timing.

Further, according to the embodiment, the information processing devicemay further include a posture detection unit configured to detect aposture of the information processing device. The user operation may bean operation performed by changing the posture of the informationprocessing device. The controller may cause the notification informationfor issuing a notification that the user operation based on the changeis accepted to be output when the posture of the information processingdevice is changed. As a result, when the posture of the informationprocessing device is changed, it is possible to display the notificationinformation used to issue a notification that the user operation basedon the change is accepted.

Further, according to the embodiment, when the posture of theinformation processing device is changed, the controller may cause thenotification information to be output by superimposing an additionalimage visually indicating the change on an image displayed on thedisplay unit. As a result, when the posture of the informationprocessing device is changed, the additional image visually indicatingthe change can be displayed by superimposing it on an image displayed onthe display unit.

Further, according to a second embodiment of the present technology,there is provided a communication system, an information processingmethod of the same and a program for causing a computer to perform themethod, the communication system including a first informationprocessing device, and a second information processing device. The firstinformation processing device causes an image based on image datareceived from the second information processing device using short-rangewireless communication to be displayed on a first display unit, andtransmits control data corresponding to a user operation related to theimage to the second information processing device when the useroperation is accepted. The second information processing devicetransmits image data that is used to display the image on the firstinformation processing device using the short-range wirelesscommunication while causing an image identical with the image to bedisplayed on a second display unit and transmitting data indicating thatthe control data is received to the first information processing devicewhen the control data is received from the first information processingdevice. At least one of a first operation and a second operation isperformed, the first operation allowing the first information processingdevice to superimpose an additional image indicating that the useroperation is accepted on the image displayed on the first display unit,the second operation allowing the second information processing deviceto superimpose an additional image indicating that the control data isaccepted on the image displayed on the second display unit. As a result,it is possible to perform at least one of the first operation allowingthe first information processing device to cause an additional imageindicating that the user operation is accepted to be displayed bysuperimposing the additional image on the image displayed on the firstdisplay unit and the second operation allowing the second informationprocessing device to cause an additional image indicating that thecontrol data is accepted to be displayed by superimposing the additionalimage on the image displayed on the second display unit.

According to one or more of embodiments of the present technology, it ispossible for the user to easily recognize the acceptance condition ofthe user operation.

BRIEF DESCRIPTION OF THE DRAWINGS

FIG. 1 is a diagram illustrating an exemplary configuration of acommunication system 100 according to a first embodiment of the presenttechnology;

FIG. 2 is a block diagram illustrating an exemplary functionalconfiguration of an information processing device 200 according to thefirst embodiment of the present technology;

FIG. 3 is a block diagram illustrating an exemplary functionalconfiguration of an information processing device 300 according to thefirst embodiment of the present technology;

FIGS. 4A and 4B are diagrams illustrating a display transition exampleof a display unit 350 in a case where the user operates an informationprocessing device 300 according to the first embodiment of the presenttechnology;

FIGS. 5A and 5B are diagrams illustrating a display transition exampleof the display unit 350 in a case where the user operates theinformation processing device 300 according to the first embodiment ofthe present technology;

FIGS. 6A and 6B are diagrams illustrating a display transition exampleof the display unit 350 in a case where the user operates theinformation processing device 300 according to the first embodiment ofthe present technology;

FIGS. 7A and 7B are diagrams illustrating a display transition exampleof the display unit 350 in a case where the user operates theinformation processing device 300 according to the first embodiment ofthe present technology;

FIGS. 8A and 8B are diagrams illustrating a display transition exampleof the display unit 350 in a case where the user operates theinformation processing device 300 according to the first embodiment ofthe present technology;

FIGS. 9A and 9B are diagrams illustrating a display transition exampleof the display unit 350 in a case where the user operates theinformation processing device 300 according to the first embodiment ofthe present technology;

FIG. 10 is a sequence chart illustrating an exemplary communicationprocess to be performed in each device constituting the communicationsystem 100 according to the first embodiment of the present technology;

FIG. 11 is a flowchart illustrating an example of a processing procedureof a data communication control process to be performed by theinformation processing device 200 according to the first embodiment ofthe present technology;

FIG. 12 is a flowchart illustrating an example of a processing procedureof a data communication control process to be performed by theinformation processing device 300 according to the first embodiment ofthe present technology;

FIG. 13 is a sequence chart illustrating an exemplary communicationprocess to be performed in each device constituting the communicationsystem 100 according to a second embodiment of the present technology;

FIG. 14 is a flowchart illustrating an example of a processing procedureof a data communication control process to be performed by theinformation processing device 200 according to the second embodiment ofthe present technology;

FIG. 15 is a flowchart illustrating an example of a processing procedureof a data communication control process to be performed by theinformation processing device 300 according to the second embodiment ofthe present technology;

FIG. 16 is a diagram illustrating an exemplary display of eachinformation processing device constituting a communication system 100according to a third embodiment of the present technology;

FIG. 17 is a block diagram illustrating an exemplary functionalconfiguration of an information processing device 201 according to amodified example of the present technology;

FIG. 18 is a diagram illustrating an exemplary configuration of acommunication system 700 according to a modified example of the presenttechnology; and

FIG. 19 is a diagram illustrating an exemplary configuration of acommunication system 800 according to a modified example of the presenttechnology.

DETAILED DESCRIPTION OF THE EMBODIMENT(S)

Hereinafter, preferred embodiments of the present disclosure will bedescribed in detail with reference to the appended drawings. Note that,in this specification and the appended drawings, structural elementsthat have substantially the same function and structure are denoted withthe same reference numerals, and repeated explanation of thesestructural elements is omitted.

Embodiments for implementing the present technology (hereinafter simplyreferred to as “embodiment”) will be described below. In addition, thedescription will be made in the following order.

1. First Embodiment (display control: an example of displayingadditional image when user operation is accepted and erasing additionalimage based on delay time)2. Second Embodiment (display control: an example of displayingadditional image when user operation is accepted and erasing additionalimage based on data from transmission-side information processingdevice)3. Third Embodiment (display control: an example of displayingadditional image on both information processing devices of reception andtransmission sides)

4. Modified Example 5. Modified Example 1. First Embodiment ExemplaryConfiguration of Communication System

FIG. 1 is a diagram illustrating an exemplary configuration of acommunication system 100 according to a first embodiment of the presenttechnology. In FIG. 1, there is illustrated an example of a systemconfiguration when two information processing devices (informationprocessing devices 200 and 300) are directly connected to each other ina wireless manner.

The communication system 100 includes the information processing devices200 and 300. In addition, the information processing devices 200 and 300each has a short-range wireless communication function. Theseinformation processing devices are, for example, wireless communicationdevices based on the IEEE (Institute of Electrical and ElectronicsEngineers) 802.11 standard. Thus, the information processing devices 200and 300 can exchange various types of information using the short-rangewireless communication function.

Here, for example, a wireless LAN (Local Area Network) and Wi-Fi Directcan be used as short-range wireless communication to be used in thecommunication system 100. In addition, for example, TDLS (TunneledDirect Link Setup) and an Ad-hoc network can be used as short-rangewireless communication to be used in the communication system 100.Additionally, for example, Wi-Fi CERTIFIED Miracast can be used asshort-range wireless AV (Audio Visual) transmission communication to beused in the communication system 100. The Wi-Fi CERTIFIED Miracast is amirroring technology in which audio or video to be played back in oneterminal is sent to another terminal and the other terminal outputs theaudio or video data as well, by using Wi-Fi Direct or TDLS standard.

Furthermore, as an example, the communication system 100 may beimplemented by applying a user interface back channel (UIBC) to Wi-FiCERTIFIED Miracast. The UIBC is, for example, a protocol used inexchange between Host PC (host computer) and USB (Universal Serial Bus).In the Wi-Fi CERTIFIED Miracast standard, the UIBC is implemented on aTCP IP (Transmission Control Protocol/Internet Protocol). In addition,instead of Wi-Fi CERTIFIED Miracast, VNC (Virtual Network Computing) maybe applied to the communication system 100.

Moreover, the information processing device 200, for example, may be adisplay device (for example, televisions, personal computers) orportable information processing device (for example, cameras, tabletterminals, smart phones) equipped with a wireless communicationfunction. In addition, the information processing device 300, forexample, may be a portable information processing device equipped with awireless communication function (for example, tablet terminals, smartphones).

In FIG. 1, there is illustrated a case where the communication system100 is used by a user 10 in a state where the information processingdevice 300 is being held by his hand. As an example, the informationprocessing device 200 can transmit image data using short-range wirelesscommunication, and the information processing device 300 can receive theimage data for outputting it. In addition, the user 10 can performvarious kinds of operations on the information processing device 300held in his hand.

Exemplary Configuration of Information Processing Device

FIG. 2 is a block diagram illustrating an exemplary functionalconfiguration of the information processing device 200 according to thefirst embodiment of the present technology.

The information processing device 200 includes an antenna 210, awireless communication unit 220, an encoder 230, an operation acceptanceunit 240, a display unit 250, and a controller 260.

The wireless communication unit 220 performs transmission or receptionof various types of information (for example, image data) to and fromanother information processing device (for example, the informationprocessing device 300) via the antenna 210 by using short-range wirelesscommunication. As an example, when a transmission process is performed,the encoder 230 compresses image data (video data) output from thecontroller 260 and the compressed image data is transmitted from theantenna 210 through the wireless communication unit 220.

The encoder 230 compresses (encodes) image data to be transmitted andoutputs the compressed image data to the controller 260. In addition,the encoder 230 may be implemented by executing an encoding process insoftware or may be implemented by executing an encoding process inhardware. Additionally, the image data to be a transmission object maybe transmitted as raw data without being encoded by the encoder 230.

The operation acceptance unit 240 accepts an operation input performedby the user and outputs operation information corresponding to theaccepted operation input to the controller 260.

The display unit 250 displays various types of information (images,games, etc.) under the control of the controller 260. In addition, thedisplay unit 250, for example, may be a display panel such as organic EL(Electro Luminescence) panel, LCD (Liquid Crystal Display) panel, or thelike. Additionally, the operation acceptance unit 240 and the displayunit 250 may be integrally configured as a touch panel on which the usercan perform an operation input by bring his finger into contact with orin proximate to a display surface.

The controller 260 is configured to control each unit of the informationprocessing device 200. As an example, the controller 260 controls thetransmission of image data used to display an image to the informationprocessing device 300. In addition, for example, when control datacorresponding to a user operation accepted in the information processingdevice 300 is received from the information processing device 300, thecontroller 260 performs a process based on the received control data.

Exemplary Configuration of Information Processing Device

FIG. 3 is a block diagram illustrating an exemplary functionalconfiguration of the information processing device 300 according to thefirst embodiment of the present technology.

The information processing device 300 includes an antenna 310, awireless communication unit 320, a decoder 330, an operation acceptanceunit 340, a display unit 350, a posture detection unit 360, a delay timecalculation unit 370, and a controller 380.

The wireless communication unit 320 performs transmission or receptionof various types of information (for example, image data) to and fromanother information processing device (for example, the informationprocessing device 200) via the antenna 310 by using short-range wirelesscommunication. As an example, when a reception process is performed, thedecoder 330 decodes image data received through the wirelesscommunication unit 320 by the antenna 310, and the decoded image data issupplied to the controller 380. The controller 380 then causes thedisplay unit 350 to display an image corresponding to the decoded imagedata.

The decoder 330 decodes image data output from the informationprocessing device 200 and outputs the decoded image data to thecontroller 380. In addition, the decoder 330 may be implemented byexecuting an encoding process in software or may be implemented byexecuting an encoding process in hardware. Additionally, if image datatransmitted from the information processing device 200 is raw data, theraw data may be used without being decoded by the decoder 330.

The operation acceptance unit 340 accepts an operation input performedby the user and outputs operation information corresponding to theaccepted operation input to the controller 380. As an example, when adisplay screen based on image data received from the informationprocessing device 200 using short-range wireless communication isdisplayed on the display unit 350, the operation acceptance unit 340accepts a user operation related to the display screen. As an example,the operation acceptance unit 340 may be implemented as a touch panel, akeyboard, a mouse.

The display unit 350 is configured to display various types ofinformation (images, games, etc.) under the control of the controller380. The display unit 350 may display an image (for example, displayscreens shown FIGS. 4A, 4B, 5A, 5B, 6A, 6B, 7A, 7B, 8A, 8B, 9A, and 9B)based on image data transmitted from the information processing device200. In addition, the display unit 350 may be a display panel such asorganic EL panel, LCD panel, or the like. Additionally, in theembodiment of the present technology, there is illustrated an example inwhich the operation acceptance unit 340 and the display unit 350 areintegrally configured as a touch panel on which the user can perform anoperation input by bring his finger into contact with or in proximate toa display surface.

The posture detection unit 360 detects the change in posture of theinformation processing device 300 by detecting acceleration, motion,tilt, or the like of the information processing device 300 and outputsposture information related to the detected change in posture to thecontroller 380. It is preferable that a sensor capable of detecting therotation angle of three axes (for example, X, Y, and Z axes) in theinformation processing device 300 is used as the posture detection unit360. For example, the posture detection unit 360 may be an accelerationsensor capable of detecting the direction of acceleration, a tilt sensorfor detecting tilt of a device, and an angular velocity sensor or gyrosensor capable of detecting rotational movement.

The delay time calculation unit 370 is configured to calculate a delaytime in short-range wireless communication generated between theinformation processing device 200 and the information processing device300. The delay time calculation unit 370 outputs the calculated delaytime to the controller 380. In addition, a method of calculating thedelay time will be described in detail with reference to FIG. 10.

The controller 380 controls each unit of the information processingdevice 300. As an example, the controller 380 controls the display unit350 to display a display screen based on image data received from theinformation processing device 200. In this case, when accepting a useroperation, the controller 380 controls the display unit 350 to displaynotification information for issuing a notification that a useroperation is accepted. As an example of the notification information, anadditional image which indicates that the user operation is accepted(for example, a ripple image 402 shown in FIG. 4B) is displayed bysuperimposing the additional image on a display screen (a display screenbased on image data received from the information processing device 200)displayed on the display unit 350. Additionally, when a user operationis accepted, the controller 380 controls to transmit control datacorresponding to the user operation to the information processing device200.

Furthermore, for example, the controller 380 causes the additional imagedisplayed on the display unit 350 to be erased at a predeterminedtiming. As an example, the controller 380 can determine thepredetermined timing based on a delay time calculated by the delay timecalculation unit 370.

Exemplary Display of Reception-Side Information Processing Device

FIGS. 4A, 4B, 5A, 5B, 6A, 6B, 7A, 7B, 8A, 8B, 9A, and 9B are diagramsillustrating a display transition example of the display unit 350 in acase where the user operates the information processing device 300according to the first embodiment of the present technology.

Exemplary Display when Performing Pressing Operation

In FIG. 4A, there is illustrated an exemplary display in a case where acontent list screen 400 is displayed on the display unit 350 of theinformation processing device 300. The content list screen 400 isdisplayed based on image data transmitted from the informationprocessing device 200. As an example, a representative image thatrepresents content (for example, moving image content or still imagecontent) is displayed on the content list screen 400 for each content.The user can use desired content by performing an operation (forexample, a touch operation by the user) of selecting a representativeimage corresponding to the desired content in the content list screen400.

Here, in a case where only a display screen based on image datatransmitted from the information processing device 200 is displayed onthe display unit 350, it is assumed that the user performs an operationinput (user operation) while viewing this display screen. In this case,for example, when a user operation is performed on the content listscreen 400, control data corresponding to the user operation istransmitted from the information processing device 300 to theinformation processing device 200. The information processing device200, when receiving the control data, performs a process correspondingto the control data and then transmits image data used to display adisplay screen after performing the process (a display screen in whichthe control data is reflected) to the information processing device 300.

In this way, in a case where a user operation is performed in thecontent list screen 400, after the exchange of information between theinformation processing device 300 and the information processing device200 is performed, the display screen in which the control data isreflected is displayed. For this reason, it may take a longer time untilthe display screen reflecting the user operation is displayed. In such acase, even though a user operation is performed in the content listscreen 400, it may not be possible for the user to quickly recognizewhether the user operation is properly accepted or not, and thus itbecomes difficult to determine whether the user operation is actuallyaccepted or not. In this time, the erroneous operation, such asdetermining that the user operation is not accepted properly and thusperforming again an operation input, is more likely to occur.

Therefore, in the first embodiment of the present technology, when auser operation is performed on the display screen that is displayed onthe display unit 350, an additional image indicating that the useroperation is properly accepted (for example, a ripple image 402 shown inFIG. 4B) is immediately displayed by superimposing it on the displayscreen. This makes it possible to quickly notify the user that the useroperation is accepted.

In FIG. 4B, there is illustrated an exemplary display immediately aftera user operation is performed on the content list screen 400. In thisexample, there is illustrated an exemplary display in a case where auser operation for selecting a representative image 401 is performed onthe content list screen 400.

As illustrated in FIG. 4B, in the content list screen 400, in a casewhere a user operation for selecting the representative image 401 (forexample, a pressing operation) is performed, the controller 380 causes aripple image (additional image) 402 to be displayed around the selectedrepresentative image 401. The controller 380 then transmits control data(including a pressed position and time) indicating that therepresentative image 401 is selected to the transmission-sideinformation processing device 200. In this way, the ripple image 402indicating that the user operation for selecting the representativeimage 401 is properly accepted is displayed immediately after the useroperation is performed, and thus it is possible for the user torecognize easily and quickly that the user's own operation is accepted.

Here, in FIG. 4B, there is illustrated an example where the ripple image402 of a wave pattern on which a plurality of colors are imparted isdisplayed as an additional image. However, a semi-transparent rippleimage may be displayed as an additional image. In addition, an image ofother shapes may be displayed as an additional image.

Moreover, in FIG. 4B, there is illustrated an example of performing apressing operation as a user operation. However, even when otherpressing operations (for example, long press, double click) areperformed, the ripple image may be similarly displayed as an additionalimage. In this case, the color or shape of the ripple image may bechanged according to the type of user operations. In addition, forexample, when a long press operation is performed as a user operation,the color or size may be changed according to the length of a pressingtime. Additionally, for other pressing operations, an image of othershape may be displayed as an additional image.

Furthermore, an additional image may be similarly displayed even whenperforming each operation (for example, tracing operation,enlargement/reduction operation) other than a pressing operation. Thesedisplay examples will be illustrated in FIGS. 5A, 5B, 6A, 6B, 7A, and7B.

Exemplary Display when Performing Tracing Operation

In FIG. 5A, there is illustrated an exemplary display in a case where anoperation screen 410 is displayed on the content list screen 400 shownin FIGS. 4A and 4B. The operation screen 410 is displayed based on imagedata transmitted from the information processing device 200, forexample, according to a predetermined timing or a user operation.

An operation region image 411 and an operation indicator 412 aredisplayed on the operation screen 410. The operation region image 411 isused to perform settings (for example, a display modification setting)of each process related to the content list screen 400 by moving theoperation indicator 412.

The operation indicator 412 can be moved by a touch operation or tracingoperation of the user in the operation region image 411. For example, ina state where a portion on which the display indicator 412 is displayedin the operation region image 411 is touched with the user's finger, theuser moves his finger to a desired position in the operation regionimage 411 and thus a moving operation (scroll operation) of theoperation indicator 412 can be performed. In addition, the user can movethe operation indicator 412 to the position where his finger is touchedby performing the operation (touch operation) in which the user touchesthe desired position in the operation region screen 411 with his finger.

In FIG. 5B, there is illustrated an exemplary display of the operationscreen 410 immediately after performing the user operation. In thisexample, there is illustrated an exemplary display in a case where auser operation for moving the operation indicator 412 from the left tothe right is performed in the operation region image 411.

As illustrated in FIG. 5B, when a user operation (for example, tracingoperation) for moving the operation indicator 412 from the left to theright is performed in the operation region image 411, the controller 380allows ripple images (additional images) 415 and 416 indicating thefirst and last positions, respectively, to be displayed. In addition,the controller 380 causes a trajectory image (additional image) 417indicating a trajectory of the movement to be displayed. The controller380 transmits control data to the transmission-side informationprocessing device 200. This control data indicates that the tracingoperation for moving the operation indicator 412 from the left to theright is performed (including a touched position and a trajectorythereof). In this way, immediately after a user operation for moving theoperation indicator 412 from the left to the right is performed, theadditional images (the ripple images 415 and 416, and the trajectoryimage 417) indicating that the user operation is properly accepted aredisplayed. This makes it possible for the user to recognize easily andquickly that the user's own operation is accepted.

In a similar way as FIG. 4B, instead of each additional image shown inFIG. 5B, images of other display forms may be displayed as an additionalimage.

Exemplary Display when Performing Reduction Operation

In FIG. 6A, there is illustrated an exemplary display in a case where acontent display screen 420 is displayed. The content display screen 420is the display screen (for example, a screen on which moving or stillimages are played back) which is displayed after performing theselection operation of the representative image 401 shown in FIGS. 4Aand 4B. The content display screen 420 is displayed based on image datatransmitted from the information processing device 200.

Here, as shown by arrows 421 and 422, it is assumed that a useroperation for reducing an image displayed on the content display screen420 (for example, an operation in which an image is pinched with twofingers of the hand 20 (pinch)) is performed. In this case, asillustrated in FIG. 6B, arrow images (additional images) 425 and 426according to the user operation are displayed.

In FIG. 6B, there is illustrated an exemplary display of the contentdisplay screen 420 immediately after performing the user operation. Inthis example, there is illustrated an exemplary display in a case wherethe user operation shown in FIG. 6A is performed. In addition, the hand20 is omitted in FIG. 6B, for simplicity and clarity of illustration.

As illustrated in FIG. 6B, when a user operation for reducing an imagedisplayed on the content display screen 420 is performed, the controller380 causes the arrow images (additional images) 425 and 426 whichrespectively indicate the first and last positions of the image to bedisplayed. The controller 380 transmits control data indicating that auser operation for reducing an image displayed on the content displayscreen 420 (including two touched positions and trajectory thereof) isperformed to the transmission-side information processing device 200.

Here, immediately after a user operation is performed on the contentdisplay screen 420, image data in which the user operation is reflectedhas not been transmitted from the information processing device 200.Thus, as illustrated in FIG. 6B, the arrow images (additional images)425 and 426 are displayed on the display screen when the user operationis performed. In other words, the arrow images (additional images) 425and 426 are displayed in a state before performing the reduction processof an image corresponding to the user operation. After that, when imagedata in which the user operation is reflected is transmitted from theinformation processing device 200, a display screen on which a reductionprocess corresponding to the user operation is performed is displayed.

In this way, immediately after performing a user operation for reducingan image displayed on the content display screen 420, the arrow images(additional images) 425 and 426 indicating that the user operation isproperly accepted are displayed. Thus, it is possible for the user torecognize easily and quickly that the user's own operation is accepted.

Exemplary Display when Performing Enlargement Operation

In FIG. 7A, there is illustrated an exemplary display in a case wherethe content display screen 420 is displayed. In FIG. 7B, there isillustrated an exemplary display immediately after performing a useroperation for enlarging an image displayed on the content display screen420.

Furthermore, in the example illustrated in FIGS. 7A and 7B, anenlargement operation is performed instead of performing the reductionoperation in the example illustrated in FIGS. 6A and 6B, and thusdetailed explanation thereof is omitted. In other words, the example ofFIGS. 7A and 7B illustrate a case where a user operation for enlargingan image displayed on the content display screen 420 (for example, anoperation in which an image is pinched with two fingers of the hand 20(pinch)) is performed as illustrated by arrows 431 and 432. In thiscase, as illustrated in FIG. 7B, arrow images (additional images) 435and 436 are displayed according to the user operation.

In this way, when a contact operation (or a proximity operation) isaccepted on a display surface (a touch panel) of the display unit 350,the controller 380 causes an additional image to be displayed by using aposition in the display surface in which the contact operation (or aproximity operation) is accepted as a reference. For example, when apressing operation that presses the representative image (operationimage) 401 is accepted as illustrated in FIG. 4B, the controller 380causes the ripple image 402 centered on a position in the displaysurface in which the pressing operation is accepted to be displayed asan additional image. In addition, as an example, when an operation oftracing an operation image in a display surface is accepted asillustrated in FIGS. 5A, 5B, 6A, 6B, 7A, and 7B, the controller 380causes an image indicating a trajectory of the tracing operation in thedisplay surface to be displayed as an additional image. The imageindicating the trajectory is, for example, the ripple images 415, 416and trajectory image 417 shown in FIG. 5B, the arrow images 425 and 426shown in FIG. 6B, and the arrow images 435 and 436 shown in FIG. 7B.

Exemplary Display when Performing Operation by Tilting InformationProcessing Device

In FIG. 8A, there is illustrated an exemplary display in a case where acontent display screen 440 is displayed. The content display screen 440is, for example, the display screen displayed based on a user operation.The content display screen 440 is displayed based on image datatransmitted from the information processing device 200. The contentdisplay screen 440 is, for example, a display screen for performing agame of operating a car 441 while avoiding an obstacle (cows, stumps) onthe road. As an example, it is assumed that the car 441 can be turned tothe left by tilting the information processing device 300 to the leftand the car 441 can turned to the right by tilting the informationprocessing device 300 to the right.

Here, it is assumed that an operation of tilting the informationprocessing device 300 to the right (operation of turning the car 441 tothe right) is performed. In this case, as illustrated in FIG. 8B, anarrow image (additional image) 442 corresponding to the user operationis displayed.

In FIG. 8B, there is illustrated an exemplary display immediately afterperforming a user operation in the content display screen 440. In thisexample, there is illustrated an exemplary display in a case where theoperation of tilting the information processing device 300 to the right(the operation of turning the car 441 to the right) is performed. Inaddition, a hand of the user is omitted in FIGS. 8A and 8B, forsimplicity and clarity of illustration.

As illustrated in FIG. 8B, when the operation of tilting the informationprocessing device 300 to the right (the operation of turning the car 441to the right) is performed, the controller 380 can detect the operationbased on posture information (information related to the change inposture) output from the posture detection unit 360. In this way, whenthe operation is detected, the controller 380 causes the arrow image(additional image) 442 indicating the tilting direction to be displayed.The controller 380 then transmits control data (including the tiltingdirection and angle thereof) indicating that a user operation of tiltingthe information processing device 300 to the right (the operation ofturning the car 441 to the right) is performed to the transmission-sideinformation processing device 200.

Here, immediately after performing a user operation in the contentdisplay screen 440, image data in which the user operation is reflectedhas not been transmitted from the information processing device 200.Thus, as illustrated in FIG. 8B, the arrow image (additional image) 442is displayed on an image when the user operation is performed. In otherwords, the arrow image (additional image) 442 is displayed in a statebefore the car 441 is moved according to the user operation. After that,when the image data in which the user operation is reflected istransmitted from the information processing device 200, a display screenon which the car 441 is moved according to the user operation isdisplayed.

In this way, immediately after performing the operation of tilting theinformation processing device 300 to the right (the operation of turningthe car 441 to the right), the arrow image (additional image) 442indicating that the user operation is properly accepted is displayed.This makes it possible for the user to recognize easily and quickly thatthe user's own operation is accepted.

Exemplary Display when Performing Operation by Rotating InformationProcessing Device

In FIG. 9A, there is illustrated an exemplary display in a case where acontent display screen 450 is displayed. The content display screen 450is, for example, a display screen displayed based on a user operationand is displayed based on image data transmitted from the informationprocessing device 200. The content display screen 450 is, for example, adisplay screen for playing back an animation. In addition, theinformation processing device 300 is rotated by an angle of ninetydegrees by using an axis perpendicular to a display surface of thedisplay unit 350 as a rotation axis, and thus it is assumed that it ispossible to change between the longitudinal state and lateral state ofthe display unit 350.

Here, as shown by an arrow 451, it is assumed that an operation ofrotating the information processing device 300 in the clockwise (CW)direction by an angle of ninety degrees by using an axis perpendicularto a display surface of the display unit 350 as a rotation axis isperformed. In this case, as illustrated in FIG. 9B, an arrow image(additional image) 452 is displayed according to a user operation.

In FIG. 9B, there is illustrated an exemplary display immediately afterperforming a user operation in the content display screen 450. In thisexample, there is illustrated an exemplary display in a case where anoperation of rotating the information processing device 300 in the CWdirection by an angle of ninety degrees is performed. In addition, ahand of the user is omitted in FIGS. 9A and 9B for simplicity andclarity of illustration.

As illustrated in FIG. 9B, when an operation of rotating the informationprocessing device 300 in the CW direction by an angle of ninety degreesis performed, the controller 380 can detect the operation based onposture information (information related to the change in posture)output from the posture detection unit 360. In this way, when theoperation is detected, the controller 380 causes the arrow image (anadditional image) 452 indicating the direction of the rotation to bedisplayed. The controller 380 then transmits control data to thetransmission-side information processing device 200. This control data(including the rotation direction and angle thereof) indicates that theoperation of rotating the information processing device 300 in the CWdirection by an angle of ninety degrees is performed.

Here, immediately after performing a user operation in the contentdisplay screen 450, image data in which the user operation is reflectedhas not been transmitted from the information processing device 200.Thus, as illustrated in FIG. 9B, the arrow image (additional image) 452is displayed on the image when the user operation is performed. In otherwords, the arrow image (additional image) 452 is displayed in a statebefore the image is rotated according to the user operation. After that,when the image data in which the user operation is reflected istransmitted from the information processing device 200, a display screenon which the image is rotated according to the user operation isdisplayed.

In this way, immediately after performing the operation of rotating theinformation processing device 300 in the CW direction by an angle ofninety degrees, the additional image 452 indicating that the useroperation is properly accepted is displayed. This makes it possible forthe user to recognize easily and quickly that the user's own operationis accepted.

Thus, when a posture of the information processing device 300 ischanged, the controller 380 performs the control of outputtingnotification information which is used to issue a notification that auser operation based on the change (an operation performed by changing aposture of the information processing device 300) is accepted. In thiscase, as an example, the controller 380 can output the notificationinformation by causing an additional image (the arrow image 442 shown inFIGS. 8A and 8B, and the arrow image 452 shown in FIGS. 9A and 9B) thatvisually represents the change to be displayed by superimposing theadditional image on the image displayed on the display unit 350.

In a similar way to FIG. 4B and FIG. 5B, instead of each additionalimage shown in FIG. 6B, FIG. 7B, FIG. 8B, and FIG. 9B, images of otherdisplay modes may be displayed as an additional image. For example, thearrow images (additional images) 442 and 452 shown in FIG. 8B and FIG.9B may be displayed on the entire display screen in a state that theimage is thin and blinked. [Communication Example]

FIG. 10 is a sequence chart illustrating an exemplary communicationprocess performed in each device constituting the communication system100 according to the first embodiment of the present technology.Additionally, in this exemplary communication process, there isillustrated a communication process in a case where an operation inputby the user is performed in a simplified way.

Initially, a network connection process is performed between thetransmission-side information processing device 200 and thereception-side information processing device 300 (601 and 602). In otherwords, a connection setup is performed between the transmission-sideinformation processing device 200 and the reception-side informationprocessing device 300 (601 and 602).

Subsequently, a capability exchange process of exchanging capabilitiesin the communication between the transmission-side informationprocessing device 200 and the reception-side information processingdevice 300 (the process of exchanging, for example, codec informationand any other supporting data format information) is performed (603 and604). In this capability exchange process, for example, each type ofinformation indicating a compression format of image data, size of imageused to transmit data, and kind of control data capable of beingtransmitted is exchanged (603 and 604). As an example, when Wi-FiCERTIFIED Miracast is used, the above-described data can be exchangedand data format to be transmitted can be determined by using a RTSP(Real Time Streaming Protocol) packet. In addition, a capability to beused may be previously determined between devices, and image data may betransmitted on the previously determined capability. In this case, thecapability exchange process may be omitted.

Subsequently, a type of codec to be used (Video Codec) in each of thetransmission-side information processing device 200 and thereception-side information processing device 300 is determined (605 and606). Next, the transmission-side information processing device 200generates image data corresponding to a display screen (UI screen) whichis to be displayed on the reception-side information processing device300 (607). The transmission-side information processing device 200 thenencodes the generated image data and transmits the encoded image data tothe reception-side information processing device 300 (608 and 609). Asan example, when Wi-Fi CERTIFIED Miracast is used, image data istransmitted by using RTP (Real-time Transport Protocol) data.

When the image data transmitted from the transmission-side informationprocessing device 200 is received (609), the reception-side informationprocessing device 300 decodes the received image data and causes thedisplay unit 350 to display a display screen corresponding to thedecoded image data (610). As an example, the content list screen 400shown in FIG. 4A is displayed on the display unit 350.

Subsequently, the delay time calculation unit 370 of the reception-sideinformation processing device 300 calculates an approximate delay timebetween the transmission-side information processing device 200 and thereception-side information processing device 300 (611). As an example,the delay time is calculated based on an RTT (Round Trip Time) value ofpacket data exchanged between each device or the size of a bufferincluded in the reception-side information processing device 300 (611).Here, the RTT value of packet data is the value indicating the time fromtransmission of a signal to reception of a response. In addition, forexample, a RTT value of packet data exchanged between the informationprocessing device 200 and the information processing device 300 in thecommunication of the capability exchange process (603 and 604) or thecommunication process of image data (608 and 609) can be used as the RTTvalue of packet data.

Here, an example of a calculation method of calculating a delay time(estimated delay time) is illustrated below. The delay time can becalculated by using the following Equation (1).

Delay Time=Network Delay+(Packet Arrival Time-Packet Playback Time)  (1)

In the above Equation (1), the network delay can be obtained by usingthe RTT value of packet data. In addition, the (packet arrivaltime-packet playback time) can be obtained by using the size of a bufferincluded in the information processing device 300.

The reception-side information processing device 300 holds thecalculated delay time. A value of the delay time may be updated everytime when a packet is transmitted or received. In addition, a value ofthe delay time may be an initially calculated value as a fixed value.

In the following, it is assumed that the user performs an operationinput (a user input) in a state where a display screen corresponding toimage data transmitted from the transmission-side information processingdevice 200 is displayed on the display unit 350. When an operation inputis performed in this way (612), the reception-side informationprocessing device 300 causes an additional image (effect image)indicating that the operation input is accepted to be displayed bysuperimposing the additional image on a display screen displayed on thedisplay unit 350 at the time of accepting the operation input (613). Inother words, the additional image indicating that the operation input isaccepted is displayed by superimposing it on the display screencorresponding to the image data transmitted from the transmission-sideinformation processing device 200 (overlay display). As an example, asillustrated in FIG. 4B, when a pressing operation is accepted, awave-like image (the ripple image 402) is displayed overlaid on thedisplay screen. In addition, as illustrated in FIG. 5B, when a scrolloperation is accepted, an additional image (ripple images 415 and 416,trajectory image 417) representing its operation contents is displayedin an overlaid manner.

Subsequently, the reception-side information processing device 300 setsa timer corresponding to the delay time held therein (614). This timeris a timer for measuring the timing of erasing an addition image. Inother words, an additional image continues to be displayed until thetimer that has been set expires. In the timing when the timer expires,the additional image is erased.

Subsequently, the reception-side information processing device 300transmits control data (user input data) corresponding to the operationinput to the transmission-side information processing device 200 (615and 616). As an example, the control data corresponding to the operationinput is transmitted to the transmission-side information processingdevice 200 using the UIBC protocol (615 and 616).

In this example, there is illustrated an example where the control datais transmitted (615 and 616) after displaying an additional image (613).However, an addition image may be displayed after transmitting thecontrol data.

Furthermore, in one example, when the delay time held in thereception-side information processing device 300 is smaller than areference value, an additional image may not be displayed. In otherwords, only when there is a delay to some extent, an additional imagemay be displayed and the user may be notified that a user operation isproperly accepted.

Moreover, the reception-side information processing device 300 erases anadditional image at the timing when the timer that has been set (timercorresponding to a delay time) expires (617). In other words, anadditional image is erased after the calculated delay time has elapsed(617).

Here, an image process such as fade-out may be performed for anadditional image according to a drawing time of the additional image. Asan example, an image process of allowing an additional image to bethinner according to the elapsed time from the timing of displaying anadditional image may be performed, and an additional image may be erasedwhen a predetermined time is reached.

When control data transmitted from the reception-side informationprocessing device 300 is received (616), the transmission-sideinformation processing device 200 performs a process based on thereceived control data (618). In one example, as illustrated in FIG. 4B,when the pressing operation of the representative image 401 is accepted,a process for allowing the content display screen 420 (shown in FIGS.6A, 6B, 7A and 7B) corresponding to the representative image 401 to bedisplayed on the information processing device 300 is performed as aprocess based on the control data.

Subsequently, the transmission-side information processing device 200generates image data in which the process based on the control data isreflected (619). Next, the transmission-side information processingdevice 200 encodes the generated image data and transmits the encodeddata to the reception-side information processing device 300 (620 and621). In addition, the reception-side information processing device 300decodes the image data and causes the display unit 350 to display adisplay screen corresponding to the decoded image data (622).

In the following, immediately after performing an operation input by theuser, it is assumed that an additional image indicating that theoperation input is accepted is not displayed in a superimposed manner.In this case, a result obtained by performing a process based on anoperation input by the user will reflected the timing when thereception-side information processing device 300 receives and displaysthe image data in which the process is reflected. However, the timeperiod ranging from when an operation input by the user is performed towhen a result obtained by performing a process based on the operationinput is reflected (time period ranging from 612 to 622) is oftendependent on a network delay, processing delay, or the like. As the timeperiod is longer, usability (ease of use) is likely to be reduced.

On the other hand, according to the first embodiment of the presenttechnology, an additional image indicating that an operation input bythe user is accepted is displayed in a superimposed manner immediatelyafter performing the operation input (612 and 613). Thus, it is possiblefor the user to be notified easily and quickly that the user operation(a command) is properly accepted at the timing when the operation inputby the user is performed.

Exemplary Operation of Information Processing Device

FIG. 11 is a flowchart illustrating an example of a processing procedureof a data communication control process to be performed by theinformation processing device 200 according to the first embodiment ofthe present technology. In addition, in this example, there isillustrated an example of performing short-range wireless communicationbetween the transmission-side information processing device 200 and thereception-side information processing device 300.

Initially, the controller 260 performs a network connection process withthe reception-side information processing device 300 (step S901). Inother words, a connection is set up between the transmission-sideinformation processing device 200 and the reception-side informationprocessing device 300 (step S901).

Subsequently, the controller 260 performs a capability exchange processof exchanging capabilities in the communication with the reception-sideinformation processing device 300 (step S902). Next, the controller 260determines a codec (Video Codec) to be used in short-range wirelesscommunication between the transmission-side information processingdevice 200 and the reception-side information processing device 300(step S903).

Subsequently, the controller 260 generates image data corresponding to adisplay screen (UI screen) which is to be displayed on thereception-side information processing device 300 (step S904). Next, theencoder 230 encodes the generated image data, and the controller 260transmits the encoded image data to the reception-side informationprocessing device 300 (step S905).

Subsequently, the controller 260 determines whether control datacorresponding to the operation input is received from thetransmission-side information processing device 200 (step S906). If itis determined that the control data corresponding to the operation inputis not received (NO in step S906), then the process returns to stepS904. On the other hand, If it is determined that the control datacorresponding to the operation input is received from thetransmission-side information processing device 200 (YES in step S906),then the controller 260 performs a process based on the received controldata (step S907).

Subsequently, the controller 260 determines whether an instruction toterminate the network connection is accepted (step S908). If it isdetermined that the termination instruction is accepted (YES in stepS908), then the data communication control process is ended. On theother hand, if it is determined that the termination instruction is notaccepted (NO in step S908), then the process returns to step S904. Then,the controller 260 generates image data (the image data in which theprocess based on the received control data is reflected) correspondingto a display screen (UI screen) to be displayed on the reception-sideinformation processing device 300 (step S904).

Exemplary Operation of Information Processing Device

FIG. 12 is a flowchart illustrating an example of a processing procedureof a data communication control process to be performed by theinformation processing device 300 according to the first embodiment ofthe present technology. In addition, in this example, there isillustrated an example of performing short-range wireless communicationbetween the transmission-side information processing device 200 and thereception-side information processing device 300.

Initially, the controller 380 performs a network connection processbetween the transmission-side information processing device 200 and thereception-side information processing device 300 (step S911). In otherwords, a connection is set up between the transmission-side informationprocessing device 200 and the reception-side information processingdevice 300 (step S911).

Subsequently, the controller 380 performs a capability exchange processof exchanging capabilities in the communication with thetransmission-side information processing device 200 (step S912). Next,the controller 380 determines a codec (Video Codec) to be used inshort-range wireless communication between the transmission-sideinformation processing device 200 and the reception-side informationprocessing device 300 (step S913).

Subsequently, the controller 380 determines whether image data generatedby the transmission-side information processing device 200 is receivedor not (step S914). If it is determined that the image data is notreceived (NO in step S914), then the process proceeds to step S917. Onthe other hand, if it is determined that the image data is received (YESin step S914), then the decoder 330 decodes the received image data, andthe controller 380 causes the display unit 350 to display a displayscreen corresponding to the decoded image data (step S915). In addition,step S915 corresponds to an example of the first procedure described inclaims.

Subsequently, the controller 380 calculates an approximate delay timebetween the transmission-side information processing device 200 and thereception-side information processing device 300 (step S916). Thecontroller 380 then holds the calculated delay time.

Subsequently, the controller 380 determines whether an operation inputby the user is performed or not (step S917). If it is determined thatthe operation input by the user is not performed (NO in step S917), thenthe process proceeds to step S921. On the other hand, if it isdetermined that the operation input by the user is performed (YES instep S917), then the controller 380 causes an additional imageindicating that the operation input is accepted to be displayed bysuperimposing the additional image on a display screen to be displayedon the display unit 350 at the time when the operation input is accepted(step S918). In addition, step S918 corresponds to an example of thesecond step described in claims.

Subsequently, the controller 380 sets a timer corresponding to the helddelay time (step S919). Next, the controller 380 transmits control datacorresponding to the operation input to the transmission-sideinformation processing device 200 (step S920). In addition, step S920corresponds to an example of the third step described in claims.

Subsequently, the controller 380 determines whether the timer that hasbeen set (the timer corresponding to the delay time) is expired or not(step S921). If it is determined that the timer that has been set is notexpired (NO in step S921), then the process proceeds to step S923. Onthe other hand, if it is determined that the timer that has been set isexpired (YES in step S921), then the controller 380 erases an additionalimages in which the timer is expired (step S922). In addition, if aplurality of user operations are performed, an additional image isdisplayed for each of the plurality of user operations. In this case,the timer is set for each additional image, and the additional imageshaving the expired timer are erased sequentially.

Subsequently, the controller 380 determines whether an instruction toterminate the network connection is accepted or not (step S923). If itis determined that the termination instruction is accepted (YES in stepS923), then the data communication control process is ended. On theother hand, if it is determined that the termination instruction is notaccepted (NO in step S923), then the process returns to step S914.

In this example, there is illustrated an example of transmitting controldata (S920) after displaying an additional image (step S918), but anadditional image may be displayed after transmitting control data.

In this way, in this embodiment of the present technology, when a useroperation is performed on a display screen displayed on the display unit350, an additional image indicating that the user operation is properlyaccepted (additional image corresponding to the user operation) isimmediately displayed in a superimposed manner. This makes it possibleto notify quickly the user that the user operation is accepted.

Thus, a visual feedback can be provided on the acceptance of a useroperation, and the user can be notified clearly and quickly of theacceptance of a user operation. Therefore, it is possible to provide theuser with comfortable operation feeling. In addition, the acceptance ofa user operation can be notified clearly and quickly to the user, andthus it is possible to prevent unintended operation.

Moreover, it is possible to prevent an additional image and an imageprocessed by a process based on a user operation from being overlappedby dynamically changing the time period of displaying the additionalimage according to the delay time. Thus, it is possible to prevent theimage processed by a process based on a user operation from being lessvisible. Therefore, in the embodiment of the present technology, it ispossible to allow the user to easily recognize the acceptance conditionof a user operation and it is possible to implement a user feedback inconsideration of delay caused by remote operation.

2. Second Embodiment

In the first embodiment of the present technology, there has beenillustrated the example where the reception-side information processingdevice calculates a delay time in the communication between twoinformation processing devices and erases an additional image based onthe delay time. However, the reception-side information processingdevice may erase an additional image based on information provided fromthe transmission-side information processing device.

In the following, in the second embodiment of the present technology,there will be illustrated an example where a reception-side informationprocessing device may erase an additional image based on informationprovided from a transmission-side information processing device. Acommunication system (information processing devices constituting thecommunication system) according to the second embodiment of the presenttechnology has a similar configuration to the communication 100(information processing devices 200 and 300) shown in FIG. 1 or thelike. Thus, the parts in this embodiment similar to those of thecommunication system 100 and the information processing devices 200 and300 are denoted with the same reference numerals, and a part of theirdescription is omitted. [Communication Example]

FIG. 13 is a sequence chart illustrating an exemplary communicationprocess to be performed by each device constituting the communicationsystem 100 according to the second embodiment of the present technology.In addition, FIG. 13 is a modified example of FIG. 10. Thus, parts inthis embodiment similar to those of FIG. 10 are denoted with the samereference numerals, and a part of their description is omitted. In thisexample, the calculation of delay time (611) and the setting of timer(614) shown in FIG. 10 are not performed.

Moreover, the transmission-side information processing device 200transmits an Ack (ACKnowledgement) for the received control data to thereception-side information processing device 300 (631 and 632) afterperforming a process based on the control data (618). In other words, afact that an operation input corresponding to the received control datais accepted is transmitted to the reception-side information processingdevice 300 (631 and 632).

Furthermore, the reception-side information processing device 300 mayerase (633) the additional image at the timing (632) of receiving theAck for the control data. However, there is a delay between thetransmission-side information processing device 200 and thereception-side information processing device 300, and thus it ispreferable to erase the additional image at the timing when the imagedata in which the process based on the control data is reflected isdisplayed.

Thus, for example, the transmission-side information processing device200 may give a sequence number that corresponds to the image data at thetime of accepting the control data to an Ack packet for the controldata. When the reception-side information processing device 300 receivesimage data corresponding to the sequence number (or, when displayingimage data corresponding to the sequence number), the reception-sideinformation processing device 300 may erase an additional image (633).Here, an Ack packet that stores the sequence number may be used bydefining its own packet.

In this way, when the controller 260 of the information processingdevice 200 receives control data corresponding to a user operation thatis accepted in the information processing device 300 from theinformation processing device 300, the controller 260 controls totransmit data indicating that the control data is received to theinformation processing device 300. The data indicating that the controldata is received, for example, is the Ack shown in 631 and 632 of FIG.13.

Furthermore, the controller 380 of the information processing device300, for example, can regard the timing when data indicating that thecontrol data is accepted is received from the information processingdevice 200 as the timing when the additional image is erased. Inaddition, the controller 380, for example, can regard the timing whenimage data corresponding to image data identification information isreceived from the information processing device 200 as the timing theadditional image is erased. The image data identification information isincluded in data indicating that the control data is accepted by theinformation processing device 200. Additionally, the controller 380, forexample, can regard the timing when an image based on image datacorresponding to image data identification information included in dataindicating that the control data is accepted by the informationprocessing device 200 is displayed as the timing the additional image iserased. It is noted that the image data identification information is,for example, a sequence number for identifying image data.

Exemplary Operation of Information Processing Device

FIG. 14 is a flowchart illustrating an example of a processing procedureof a data communication control process to be performed by theinformation processing device 200 according to the second embodiment ofthe present technology. In addition, FIG. 14 is a modified example ofFIG. 11. Thus, parts in this embodiment similar to those of FIG. 11 aredenoted with the same reference numerals, and a part of theirdescription is omitted.

After performing a process based on the received control data (stepS907), the controller 260 transmits an Ack for the control data to thereception-side information processing device 300 (step S931).

Exemplary Operation of Information Processing Device

FIG. 15 is a flowchart illustrating an example of a processing procedureof a data communication control process to be performed by theinformation processing device 300 according to the second embodiment ofthe present technology. In addition, FIG. 15 is a modified example ofFIG. 12. Thus, parts in this embodiment similar to those of FIG. 12 aredenoted with the same reference numerals, and a part of theirdescription is omitted.

After transmitting control data corresponding to the operation input(step S920), the controller 380 determines whether an Ack for thecontrol data is received or not (step S941). If it is determined thatthe Ack for the control data is received (YES in step S941), then thecontroller 380 determines whether image data corresponding to a sequencenumber included in an Ack packet for the control data is received or not(step S942). If it is determined that the image data corresponding tothe sequence number is received (YES in step S942), then the controller380 erases an additional image related to the Ack (step S943).

Furthermore, if it is determined that the Ack for the control data isnot received (NO in step S941), then the process proceeds to step S923.In addition, in the case where the Ack for the control data is received,if it is determined that the image data corresponding to the sequencenumber included in the Ack packet is not received (NO in step S942),then the process proceeds to step S923.

In this way, according to the second embodiment of the presenttechnology, it is possible to properly erase an additional image withoutcalculating the delay time in the communication between two informationprocessing devices.

3. Third Embodiment

In the first and second embodiments of the present technology, there hasbeen illustrated an example of displaying an additional image on thereception-side information processing device. However, an additionalimage may be displayed on the transmission-side information processingdevice. In addition, an additional image may be displayed on bothinformation processing devices at the reception and transmission sides.Additionally, an additional image may be displayed on any one of twoinformation processing devices at the reception and transmission sides.

In the following, in the third embodiment of the present technology,there will be illustrated an example of displaying an additional imageon two information processing devices at the reception and transmissionsides. A communication system (information processing devicesconstituting the communication system) according to the third embodimentof the present technology has a substantially similar configuration tothe communication 100 (information processing devices 200 and 300) shownin FIG. 1 or the like. Thus, parts in this embodiment similar to thecommunication system 100 and the information processing devices 200 and300 are denoted with the same reference numerals, and a part of theirdescription is omitted.

Exemplary Display of Additional Image

FIG. 16 is a diagram illustrating an exemplary display of eachinformation processing device constituting the communication system 100according to the third embodiment of the present technology. Inaddition, an exemplary display of the information processing device 300is similar to FIG. 4B. Additionally, a display screen identical with thedisplay image displayed on the display unit 350 of the informationprocessing device 300 is displayed on the display unit 250 of theinformation processing device 200.

Here, the controller 260 of the transmission-side information processingdevice 200 causes an image (content list screen 405) to be displayed onthe display unit 250. The image (content list screen 405) is identicalwith an image (content list screen 400) to be transmitted to thereception-side information processing device 300. In addition, asdescribed above, in the content list screen 400, it is assumed that auser operation (for example, a pressing operation) of selecting therepresentative image 401 is performed. In this case, the controller 380of the reception-side information processing device 300 causes a rippleimage (additional image) 402 to be displayed around the selectedrepresentative image 401. The controller 380 then transmits control dataindicating that the representative image 401 is selected to thetransmission-side information processing device 200. In addition, whenthe control data is received, the controller 260 of thetransmission-side information processing device 200 causes a rippleimage (additional image) 406 to be displayed on the content list screen405 based on the control data (for example, type and position of theuser operation). In other words, the same ripple images (additionalimages) 402 and 406 are displayed on the content list screens 400 and405, respectively, in a superimposed manner.

In this way, immediately after performing the user operation ofselecting the representative image 401, the ripple image 402 isdisplayed, and at the same time, the ripple image 406 can be displayedon the transmission-side information processing device 200 receiving thecontrol data related to the user operation. Thus, the user can recognizeclearly, quickly, and easily that his own operation is accepted throughany one of the transmission-side information processing device 200 andthe reception-side information processing device 300.

Determination as to whether the additional image is displayed on bothinformation processing devices at the reception and transmission sidesor the additional image is displayed on any one of the informationprocessing devices at the reception and transmission sides may besettable by an user operation.

In this way, the communication system 100 can be configured to performat least one of a first operation and a second operation. Here, thefirst operation is an operation which allows the information processingdevice 300 to display an additional image indicating that the useroperation is accepted by superimposing it on an image displayed on thedisplay unit 350. In addition, the second operation is an operationwhich allows the information processing device 200 to display anadditional image indicating that control data related to the useroperation is accepted by superimposing it on an image displayed on thedisplay unit 250.

4. Modified Example

An example of the transmission-side information processing deviceincluding the operation acceptance unit and the display unit has beenillustrated in FIG. 2. However, an embodiment of the present technologymay be applied to an information processing device which does notinclude an operation acceptance unit and/or a display unit.

In the following, as a modified example, an example of atransmission-side information processing device which does not includean operation acceptance unit and a display unit will be illustrated.

Exemplary Configuration of Information Processing Device

FIG. 17 is a block diagram illustrating an exemplary functionalconfiguration of an information processing device 201 according to themodified example of the present technology.

The information processing device 201 includes an antenna 210, awireless communication unit 220, an encoder 230, and a controller 260.In addition, the information processing device 201 is a modified exampleof the information processing device 200 shown in FIG. 2. Thus, parts inthis embodiment similar to those of the information processing device200 are denoted with the same reference numerals, and a part of theirdescription is omitted.

In this way, an embodiment of the present technology may be applied evenwhen the information processing device 201 which does not include anoperation acceptance unit and a display unit is regarded as atransmission-side information processing device.

5. Modified Example

In the first to third embodiments of the present technology, there hasbeen illustrated an example where two information processing devices(information processing devices 200 and 300) are directly connected toeach other in a wireless manner. However, other network configurationsmay be possible.

In the following, in this modified example, there will be illustrated anexample of other network configurations. In addition, an informationprocessing device according to the modified example has a substantiallysimilar configuration to examples shown in FIGS. 2 and 3, or the like.Thus, a part of descriptions common to the first to third embodiments ofthe present technology is omitted.

Exemplary Configuration of Communication System

FIG. 18 is a diagram illustrating an exemplary configuration of acommunication system 700 according to a modified example of the presenttechnology. In FIG. 18, there is illustrated an exemplary systemconfiguration when two information processing devices (informationprocessing devices 710 and 720) are wirelessly connected to each othervia a router 730.

The communication system 700 includes an information processing device710, an information processing device 720, and the router 730. Inaddition, the information processing device 710 corresponds to theinformation processing device 200 shown in FIG. 1, and the informationprocessing device 720 corresponds to the information processing device300 shown in FIG. 1.

Furthermore, although FIG. 18 illustrates an example where twoinformation processing devices (the information processing devices 710and 720) are wirelessly connected using the router 730, two informationprocessing devices may be connected to each other using other relaydevices. As an example, two information processing devices (theinformation processing devices 710 and 720) may be connected to eachother via an access point (wireless LAN access point).

Exemplary Configuration of Communication System

FIG. 19 is a diagram illustrating an exemplary configuration of acommunication system 800 according to a modified example of the presenttechnology. In FIG. 19, there is illustrated an example of a systemconfiguration in a case where two information processing devices(information processing devices 810 and 820) are wirelessly connected toeach other via the Internet 850.

The communication system 800 includes an information processing device810, an information processing device 820, routers 830 and 840, and theInternet 850. In addition, the information processing device 810corresponds to the information processing device 200 shown in FIG. 1,and the information processing device 820 corresponds to the informationprocessing device 300 shown in FIG. 1.

Furthermore, although FIG. 19 illustrates an example where twoinformation processing devices (the information processing devices 810and 820) are wirelessly connected via the Internet 850, two informationprocessing devices may be wirelessly connected to each other via othernetworks (for example, public line networks).

As illustrated in FIG. 19, an image in a remote location can becontrolled via the Internet 850. Thus, the embodiment of the presenttechnology can be implemented without depending on a network pathbetween a control terminal and an image output terminal.

Moreover, in the embodiment of the present technology, there has beenillustrated an example of visually outputting notification informationthat is used to issue a notification that a user operation is accepted(an example of displaying an additional image). However, other type ofnotification information may be output. For example, notificationinformation may be output in an audible way by outputting sound that isused to issue a notification that a user operation is accepted. Inaddition, for example, notification information may be output byvibrating the information processing device to issue a notification thata user operation is accepted.

In the embodiment of the present technology, although there has beendescribed the example of a display device or portable informationprocessing device equipped with a wireless communication function, theembodiment of the present technology can be applied to other types ofinformation processing devices. As an example, the embodiment of thepresent technology can be applied to an information processing devicesuch as home game machines, portable video processing devices, handheldgame consoles, digital still cameras, digital video cameras (forexample, a camera integrated recorder), digital photo frames. Inaddition, for example, the embodiment of the present technology can beapplied to an information processing device capable of performingwireless communications (for example, a personal computer with nowireless communication function) by attaching a wireless communicationdevice equipped with a wireless communication function.

Note that the above described embodiments show examples of embodying thepresent disclosure, and there is a correspondence between the featuresin the embodiments and the respective features of the presentdisclosure. Similarly, there is a correspondence between the features ofthe present disclosure and the respective features in the embodiments ofthe present disclosure with the same reference numerals. However, thepresent disclosure is not limited to the embodiments, and can embodyvarious modifications which do not deviate from the scope of the presentdisclosure.

Further, the procedures described in each of the aforementionedembodiments may be understood as a method including a series of theseprocedures, and may be understood as a program for causing a computer toexecute the series of these procedures or a recording medium storing theprogram therein. As the recording medium, for example, a hard disk, a CD(a Compact Disc), an MD (a MiniDisc), a DVD (a Digital Versatile Disk),a memory card, or Blu-ray Disc (registered trademark) can be used.

It should be understood by those skilled in the art that variousmodifications, combinations, sub-combinations and alterations may occurdepending on design requirements and other factors insofar as they arewithin the scope of the appended claims or the equivalents thereof.

Additionally, the present technology may also be configured as below.

(1) An information processing device including:

an operation acceptance unit configured to accept a user operationrelated to an image based on image data received from anotherinformation processing device using short-range wireless communicationwhen the image is displayed on a display unit; and

a controller configured to perform control in a manner that notificationinformation for issuing a notification that the user operation isaccepted is output when the user operation is accepted, and control datacorresponding to the user operation is transmitted to the otherinformation processing device.

(2) The information processing device according to (1), wherein thecontroller causes the notification information to be output bysuperimposing an additional image indicating that the user operation isaccepted on the image displayed on the display unit.

(3) The information processing device according to (2),

wherein the operation acceptance unit and the display unit include atouch panel,

wherein the user operation is a contact operation or a proximityoperation to the touch panel, and

wherein the controller causes the additional image to be displayed byusing a position in the touch panel at which the contact operation orthe proximity operation is accepted as a reference when the contactoperation or the proximity operation to the touch panel is accepted.

(4) The information processing device according to (3), wherein thecontroller causes a ripple image to be displayed as the additional imagewhen a pressing operation of pressing an operation image on the touchpanel is accepted, the ripple image being centered on a position in thetouch panel at which the pressing operation is accepted, and

wherein the controller causes an image indicating a trajectory of atracing operation in the touch panel to be displayed as the additionalimage when the tracing operation of tracing an operation image on thetouch panel is accepted.

(5) The information processing device according to any one of (2) to(4), wherein the controller causes the additional image to be erased ata predetermined timing.

(6) The information processing device according to (5), wherein thecontroller determines the predetermined timing based on a delay timeoccurred in the short-range wireless communication with the otherinformation processing device.

(7) The information processing device according to (5), wherein thecontroller regards a timing of receiving data indicating that thecontrol data is accepted from the other information processing device asthe predetermined timing.

(8) The information processing device according to (5), wherein thecontroller regards a timing of receiving image data corresponding toimage data identification information from the other informationprocessing device or a timing of displaying an image based on the imagedata as the predetermined timing, the image data identificationinformation being included in data indicating that the other informationprocessing device accepts the control data.

(9) The information processing device according to (1) or (2), furtherincluding:

a posture detection unit configured to detect a posture of theinformation processing device,

wherein the user operation is an operation performed by changing theposture of the information processing device, and

wherein the controller causes the notification information for issuing anotification that the user operation based on the change is accepted tobe output when the posture of the information processing device ischanged.

(10) The information processing device according to (9), wherein whenthe posture of the information processing device is changed, thecontroller causes the notification information to be output bysuperimposing an additional image visually indicating the change on animage displayed on the display unit.

(11) A communication system including:

a first information processing device; and

a second information processing device,

wherein the first information processing device causes an image based onimage data received from the second information processing device usingshort-range wireless communication to be displayed on a first displayunit, and transmits control data corresponding to a user operationrelated to the image to the second information processing device whenthe user operation is accepted,

wherein the second information processing device transmits image datathat is used to display the image on the first information processingdevice using the short-range wireless communication while causing animage identical with the image to be displayed on a second display unitand transmitting data indicating that the control data is received tothe first information processing device when the control data isreceived from the first information processing device, and

wherein at least one of a first operation and a second operation isperformed, the first operation allowing the first information processingdevice to superimpose an additional image indicating that the useroperation is accepted on the image displayed on the first display unit,the second operation allowing the second information processing deviceto superimpose an additional image indicating that the control data isaccepted on the image displayed on the second display unit.

(12) An information processing method including:

a first procedure of causing an image based on image data received fromanother information processing device using short-range wirelesscommunication to be displayed on a display unit;

a second procedure of causing notification information for issuing anotification that a user operation related to an image displayed on thedisplay unit is accepted to be output when the user operation isaccepted; and

a third procedure of transmitting control data corresponding to the useroperation to the other information processing device.

What is claimed is:
 1. A first information processing device,comprising: circuitry configured to: transmit image data to a secondinformation processing device via a wireless connection; wherein thesecond information processing device: displays a first image based onthe image data on a display of the second information processing device,accepts a user operation on the first image, and superimposes additionalinformation on the first image to indicate acceptance of the useroperation; receive control data, corresponding to the user operation,from the second information processing device; and execute a processbased on the control data, wherein the second information processingdevice erases the additional information after a threshold time.
 2. Thefirst information processing device according to claim 1, wherein theprocess comprises: generate a sequence number corresponding to the imagedata, based on the control data, and transmit the sequence number to thesecond information processing device.
 3. The first informationprocessing device according to claim 2, wherein the transmitted sequencenumber indicates that the control data is received.
 4. The firstinformation processing device according to claim 1, wherein the secondinformation processing device accepts the user operation based on thetransmitted image data.
 5. The first information processing deviceaccording to claim 1, wherein the additional information indicates thatthe user operation is accepted at the second information processingdevice.
 6. The first information processing device according to claim 1,further comprising: a display unit configured to display a second imagewhich is identical to the first image.
 7. The first informationprocessing device according to claim 1, wherein the image data istransmitted to the second information processing device on Real-TimeTransport Protocol via a Wi-Fi Direct connection.
 8. The firstinformation processing device according to claim 1, wherein the controldata is received via a User Interface Back Channel.
 9. The firstinformation processing device according to claim 1, further comprising:an input unit configured to detect user inputs, wherein the input unitcomprises at least one of a gyro sensor, an electronic compass, or anacceleration sensor, and wherein data from the input unit is utilized togenerate a sequence number.
 10. An information processing method,comprising: in a first information device: transmitting image data to asecond information processing device via a wireless connection; whereinthe second information processing device: displays a first image basedon the image data on a display of the second information processingdevice, accepts a user operation on the first image, and superimposesadditional information on the first image to indicate acceptance of theuser operation; receiving control data, corresponding to the useroperation, from the second information processing device; and executinga process based on the control data, wherein the second informationprocessing device erases the additional information after a thresholdtime.